Methods and systems for utilizing prediction models in healthcare

ABSTRACT

A method for providing decision support includes using a programmed computer to input a regression model specification, and to repeat the input a plurality of times to obtain and store a plurality of regression model specifications. The method further includes using the programmed computer to analyze selected regression model specifications to determine at least one of common variables and functions of common variables, to thereby determine a reduced-redundancy request for input of variables, when a plurality of the stored regression model specifications are selected for use.

BACKGROUND OF THE INVENTION

This invention relates generally to methods and apparatus for providingstatistical estimates useful for decision support, including computernetworks and software configured to provide such decision support. Themethods and apparatus described herein are particularly useful forproviding information to health care providers and medical patients, butare not limited to such environments and can be used in a wide varietyof fields in which regression analysis or statistical modeling is usedas a decision support tool.

This application hereby incorporates by reference in its entiretyanother application filed on even date by the same inventors as AttorneyDocket No. 23033-3 and entitled “System and Methods for RiskStratification of Patent Populations.”

The past decade of health services research has witnessed an explosionof prognostic models to help physicians understand the risks andbenefits of proposed medical therapies. However, the application of suchmodels to clinical practice has been limited by both their complexityand the lack of a practical mechanism for making them available at thetime of medical decision-making.

The emergence of the internet and mobile computing devices has creatednew opportunities for researchers in the health care field to translatetheir evidence-based predictive models into clinical decision aids.However, a number of barriers continue to prevent researchers fromtaking advantage of these technologies, including: (1) a requisiteexpertise in a computer programming language or application developmentenvironment; (2) the challenge of creating a system that can adapt to abroad range of clinical practice settings, each with unique device andwork-flow constraints; (3) the need for a sustainable mechanism forupdating predictive models and deploying revisions in a timely fashion;and (4) the difficulty of integrating with existing IT infrastructuresand disparate clinical information systems.

Furthermore, both the patient and the attending health care professionalshould be involved in making clinical health care treatment decisionsthat affect the patient's desired heath goals and quality of lifeconcerns. These decisions may vary depending upon the patient's age,sex, socioeconomic, demographic, clinical, and genetic or imagingcharacteristics, which generally affect the relative risk and benefitsof outcomes of medical and surgical procedures.

BRIEF DESCRIPTION OF THE INVENTION

In one aspect, the present invention provides a machine-readable mediumor media having instructions recorded thereon that are configured toinstruct a processor to input a regression model specification, and torepeat this input a plurality of times to obtain and store a pluralityof regression model specifications. The instructions also are configuredto analyze selected regression model specifications to determine atleast one of common variables and functions of common variables, tothereby determine a reduced-redundancy request for input of variables,when a plurality of the stored regression model specifications areselected for use.

In another aspect, the present invention provides a method for providingdecision support. The method includes using a programmed computer toinput a regression model specification, and to repeat the input aplurality of times to obtain and store a plurality of regression modelspecifications. The method further includes using the programmedcomputer to analyze selected regression model specifications todetermine at least one of common variables and functions of commonvariables, to thereby determine a reduced-redundancy request for inputof variables, when a plurality of the stored regression modelspecifications are selected for use.

In yet another aspect, the present invention provides a computer networkthat includes a server computer and a server module. The server computerincludes a processor and a memory. The computer network also includes afirst client computer, not necessarily different from the servercomputer. The first client computer includes a first user displaydevice, a first user input device, and a client module. The computernetwork also includes a second client computer, not necessarilydifferent from the first client computer or the server computer. Thesecond client computer has a second user display device not necessarilydifferent from the first user display device, a second user input devicenot necessarily different from the first user input device, and a secondclient module. The server module includes instruction code configured to(a) instruct the processor to communicate common regression models tothe first client module and store a regression model specificationreceived from the first client module, (b) to repeat (a) a plurality oftimes to obtain and store a plurality of the regression modelspecifications; and, (c) when a plurality of the stored regression modelspecifications are selected for use, to analyze the selected regressionmodel specifications to determine at least one of common variables andfunctions of common variables, to thereby determine a reduced-redundancyrequest for input of variables.

It will thus be appreciated that configurations of the present inventionfacilitate rapid translation of evidenced-based predictive models intorobust tools (for example, web-based tools) capable of providing visualrepresentations of predicted outcomes. When used in a medicalenvironment to provide outcome predictions at the point of patient care;configurations of the present invention can be used to rapidlydisseminate the newest knowledge to the clinical setting.

Moreover, outcomes researchers can use configurations of the presentinvention to create powerful evidence-based tools that deliver decisionsupport at the point of care for diagnostic workups and treatmentselection. In the management of acute coronary syndromes (ACS), forexample, physicians can get immediate probability estimates for outcomessuch as survival, angina frequency, or physical limitation at one yearfor all possible treatment options, as well as risk-projections forprocedural complications.

Some configurations provide a broad assortment of graphical outputoptions that facilitate the sharing of information with patients,thereby allowing physicians and patients to reach a consensus decisionregarding therapy that best meets the desires, needs, and expectationsof the patient. Some configurations can be or are useful for producing“informed consent” documents and/or documentation of discussion ofprocedural risks and benefits with patients, and/or for other uses.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a pictorial block diagram of a configuration of a computernetwork of the present invention.

FIG. 2 is a block diagram showing the logical structure of an exampleconfiguration of a server module.

FIG. 3 is a flow chart illustrating instruction steps provided by someconfigurations of the server module of FIG. 2 in some configurations ofthe present invention.

FIG. 4 is an example of a display of a visual selection of parametricregression model forms.

FIG. 5 is an example of a display of an outcome type screen.

FIG. 6 is an example of a display of a parameter name screen.

FIG. 7 is a flow chart illustrating additional instruction stepsprovided by some configurations of the server module of FIG. 2 in someconfigurations of the present invention.

FIG. 8 is an example of a display that provides a visual selection ofpreviously stored models.

FIG. 9 is an example of a display requesting parameter data.

FIG. 10 is an example of a result display provided by someconfigurations of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Technical effects of configurations of the present invention include thegeneration of visual displays and printed reports useful for decisionmaking in various fields. Thus, In some configurations of the presentinvention and referring to FIG. 1, a computer network 100 is configuredfor providing decision support. Decision support computer network 100may be used, for example, in a medical or clinical health careenvironment to identify an appropriate regimen for treatment of amedical condition. However, uses for computer network 100 are notlimited to the health field. To give but one of many examples, decisionsupport computer network 100 is useful in an advertising environment todetermine strategies for marketing products. More generally, computernetwork 100 is useful in many varied contexts in which statisticalregression models can be used to predict outcomes. To provide decisionsupport, some configurations of the present invention are configured togenerate outputs, including graphical outputs and patient reports thatincorporate such outputs.

In some configurations, computer network 100 comprises a server computer102 that executes a server module. The server module comprises softwareinstructions recorded on a machine-readable medium or media 104. Themachine readable medium or media may comprise, for example, one or morefloppy diskettes, CD-ROMs, CD-RWs, DVDs, DVD+Rs, DVD-Rs, DVD+RWs,DVD-RWs, memory devices such as USB memory sticks or other types ofmemory cards, internal readable and writable memory 106 of any ofvarious kinds, such as internal or external RAM, etc., read-only memory(ROM) 108 of any of various kinds, hard disks, optical drives, andcombinations thereof. (As used herein, “media” includes not only“removable” media, but also “non-removable” media such as primary andsecondary storage. For example, RAM, ROM, and hard disk drives areincluded as “media,” as well as the aforementioned types of media.)Server computer 102 can include devices (not specifically illustrated inFIG. 1) for reading removable media, such as a CD-ROM drive, a DVDdrive, a floppy disk drive, etc. In many configurations, server computer102 will comprise at least a readable and writeable memory 106,read-only memory 108 or non-volatile memory of a suitable type, and aprocessor 110 (e.g., a central processing unit or CPU) which may itselfcomprise one or more microprocessors, coprocessors, etc. Thus, the term“processor,” as used herein, is not literally restricted to a singleCPU. Moreover, server computer 102 may itself comprise a network of oneor more computers, as can any other device referred to as a “computer”herein.

Computer network 100 further comprises one or more first clientcomputers 112. In many configurations, it is in communication withserver computer 102 via a network 113, for example, the Internet. Inmany configurations of the present invention, Client computer 112comprises a first client module comprising software instructionsrecorded on a machine-readable medium or media 114. In manyconfigurations, client computer 112 further comprises at least areadable and writable memory 116, read-only memory 118, and a processor120 that may itself comprise one or more microprocessors, coprocessors,etc. First client computer 112 may itself comprise one or more computersin a network. First client computer 112 further comprises a first userdisplay device 122, such as a CRT display, LCD display, plasma display,and/or a hardcopy device such as a printer. First client computer 112also comprises a first user input device 124, such as a keyboard, amouse, a touch screen (which may be part of display 122), and/or atrackball, etc. First client computer 112 is not limited to desktop andlaptop computers but can include any computing device that cancommunicate over a network. For example, in some configurations, a firstclient computer 112 can be a personal digital assistant (PDA) or awireless telephone with a display screen.

Computer network 100 further comprises one or more second clientcomputers 126. In many configurations, second client computer 126 is incommunication with server computer 102 via network 113. Also in manyconfigurations, second client computer 126 comprises a second clientmodule comprising software instructions recorded on a machine-readablemedium or media 128. In many configurations, second client computer 126further comprises at least a readable and writable memory 130, read-onlymemory 132, and a processor 134 that may itself comprise one or moremicroprocessors, coprocessors, etc. Second client computer 126 mayitself comprise one or more computers in a network. Second clientcomputer 126 further comprises a second user display device 136, such asa CRT display, LCD display, plasma display, and./or a hardcopy devicesuch as a printer. Second client computer 126 also comprises a seconduser input device 138, such as a keyboard, a mouse, a touch screen(which may be part of display 136), and/or a trackball, etc.

As used herein, software instructions are said to “instruct a computerto display” information even if such information is communicated via anetwork to another computer for display on a remote display terminal. Inthis sense code running on a web server instructs a processor executingthat code to “display” a web page, even though the code actuallyinstructs the processor to communicate data via a network that allows abrowser program to instruct another computer to construct a display ofthe web page on the display of the other computer. For example, theserver module described in the examples presented herein can include aweb server and the client modules can comprise web browsers. Also, insome configurations, client computers 112 and 126 comprise laptop,desktop, or mobile computing devices or communication terminals. Thebroader scope of the phrase “instruct a computer to display” is usedbecause server computer 102 and the one or more client computers 112,126 need not necessarily be different computers. For example,communication protocols known in the art allow a server software moduleand a client software module running on a multitasking computer systemto communicate with one another on the same computer system, and thesame server software module can also communicate with a client softwaremodule running on a different computer via a network connection. Theexamples used herein assume, without loss of generality, that differentphysical computers and a network are used, as such will be the case inmany, but not all, configurations of the present invention.

The terms “display” and “accept” as used in the descriptions hereinrefer to a suitably programmed computing apparatus “displaying” or“accepting” data, not to a person “displaying” or “accepting” something.A person might, however, view the displayed data on an output device oron a page produced by an output device or supply the accepted data usingan input device.

In some configurations of the present invention, a method is provided toprovide decision support via software that comprises the server module.Some configurations of the present invention provide server modules thatutilize the ASP.NET platform available from Microsoft Corporation,Redmond, Wash. as well as MS Internet Information Services (IIS) and MSSQL Server from Microsoft Corporation for web services and data storage,respectively. A multi-tier system architecture provided in someconfigurations enables scaling of server module components as needed tomeet specific demands of a particular deployment. In addition, a modulardesign framework is provided in some configurations to facilitateextensibility and incorporation of new functionality via custom modules.In some configurations, the server module is written in C# using thecode-behind convention; except for its SQL data access components whichare stored procedures written in Transact-SQL. Configurations of thepresent invention are not limited to implementation using the toolsdescribed above. For example, configurations of the present inventioncan run on the LINUX® operating system (LINUX is a trademark of LinusTorvalds) and be built using a different suite of applications. Theselection of an appropriate operating system and suite of applicationscan be left as a design choice to one of ordinary skill in the art aftersuch a person gains an understanding of the present invention from thepresent description.

More particularly, and referring to FIGS. 2 and 3, a server module 150is provided on a machine readable medium or media. In someconfigurations of the present invention, server module 150 instructsprocessor 110 to perform a method illustrated by flow chart 200 of FIG.2. Although the flow charts provided herein are illustrative ofconfigurations of methods used herein, it will be understood that theorder of steps shown can be varied from the order illustrated in otherconfigurations of the present invention, that steps illustrated as beingseparate can be combined (e.g., various displays and requests for datacan be combined into a single output screen), and that not all stepsillustrated are necessarily required in all configurations.

A technical effect of the present invention is achieved first by a userlogging in with appropriate credentials. Server module 150 instructsprocessor 110 to display, at 202, a visual selection of parametricregression model forms, for example, on a user display device 122. Anexample of such a display is shown in FIG. 4. In some configurations,access to these features is available only to those with administrativerights.

A Visual Model Editor (VME) is used in some configurations of thepresent invention to provide the visual selection of parametricregression model forms. A VME is a software application or module thatprovides a user with a graphical user interface (GUI) that allows theuser to build, edit, and visualize statistical models. In someconfigurations, the GUI includes standard GUI elements such as windows,dialog boxes, menus, drop-down lists, radio-buttons, checkboxes, icons,etc; and the module provides functionality to define and express astatistical model using non-textual user input, such as mouse movementsand mouse clicks. In some configurations, the module also providesfunctionality to input those parts of a model definition that requirecontinuous (non-discrete) numeric values or free text (e.g., inputparameter name or displayed text). User interaction with the interfaceis achieved by one or more methods that may include, for example,pointing and clicking with a mouse, touchpad, or other input device, ortyping on a keyboard, or speaking into a microphone and using voicecommand recognition software. In some configurations, model definitionsare imported, either in part in their entirety, from all-text modelrepresentations, examples of which include, but are not limited to,Predictive Modeling Markup Language (PMML) documents and other XML-baseddocuments. Some configurations allow imported models to be edited andmodified within the VME, stored in a memory of the server computer orelsewhere, and/or re-exported in their original formats and/or otherformats.

A general regression model framework is used in some configurations forexpressing predictions. The model types can include, for example,linear, generalized linear, cumulative multinomial, generalizedmultinomial and proportional hazard models, thereby providing coveragefor major medical prognostic model types. For example, the selection mayinclude a linear regression model of the form f (x)=β₁x₁+β₂x₂+β₃x₃+ . .. ′β_(η)x_(η) as well as intrinsically non-linear forms and forms thatare linear in transformed variables, e.g., x₁=ln(y), where y is an inputvariable. Some configurations provide the ability to implement custommodel types using a built-in scripting interface. The visual display ofmodel type is associated with a drop-down list in some configurations,for example, and the visual display of model type changes depending uponthe item selected from the drop-down list. In some configurations, thedisplay is a list of parametric regression model forms. In otherconfigurations, the display comprises a rendering of mathematical modelson user display device 122.

Model types are defined in terms of a coefficient vector and an optionalcovariance matrix for calculating confidence intervals. For example, thecoefficient vector and covariance matrix are defined as follows:Coefficient vector b′=(b′_(0·)b′_(1·)b′ . . . b′_(r·)); and

${{{Covariance}\mspace{14mu} {{matrix}\mspace{14mu}\left\lbrack \begin{matrix}\bigvee_{1 \cdot 1} & \bigvee_{1 \cdot 2} & \ldots & \bigvee_{1 \cdot r \cdot} \\\bigvee_{2 - 1} & \bigvee_{2 \cdot 2 \cdot} & \ldots & \bigvee_{2 \cdot r \cdot} \\\vdots & \vdots & \; & \vdots \\\bigvee_{r \cdot 1 \cdot} & \bigvee_{r \cdot 2 \cdot} & \ldots & \bigvee_{r \cdot r \cdot}\end{matrix} \right\rbrack}\mspace{14mu} {{where}\bigvee_{i \cong j \cong}}}=={{Cov}\left( {b_{i \cdot},b_{j \cdot}} \right)}} = \begin{bmatrix}\bigvee_{i\; 1j\; 1} & \bigvee_{i\; 1j\; 2} & \ldots & \bigvee_{i\; 1{js}} \\\bigvee_{{i\; 2} - {j\; 1}} & \bigvee_{i\; 2j\; 1} & \ldots & \bigvee_{i\; 2{js}} \\\vdots & \vdots & \; & {\vdots \;} \\\bigvee_{{isj}\; 1} & \bigvee_{{isj}\; 2} & \ldots & \bigvee_{isjs}\end{bmatrix}$

In the definitions above, b_(i′)=(b_(i1) b_(i2) . . . b_(is)) is thevector of coefficients associated with the i^(th) predicator in x, withone coefficient for each state s (except the reference state). Also,v_(ikil)=Cov(b_(ik)b_(jl)) denotes the covariance between the i^(th)predictor, k^(th) state) and j^(th) predictor, l^(th) state)coefficients.

The user at client computer 112 enters a selection of a model type (forexample, a selection of one of the following model types: linear,generalized linear, cumulative multinomial, generalized, multimomial orproportional hazard model) from a menu. In some configurations, the menuis a “visual” menu. (“Visual” displays and menus include one or moreelements such as graphical illustrations, drop-down lists, selectionbuttons, checklists, etc.) Server computer 102 is programmed withinstructions by the server module to accept this selection at step 204.In some configurations, the actual form of mathematical model selectedis determined by several user-selected options, all of which comprisethe selection accepted by the server module at step 204. In suchconfigurations, the user is, in effect, “building” the mathematicalformulae as he or she goes along, as opposed to merely selected astatic, pre-defined functional form.

Next, at step 206, instructions in the server module instruct processor110 to request an outcome type and one or more regression modelparameter names and corresponding parameter types, and, in appropriatecases, limits and/or lists of possible input values. Examples ofparameter names in a medical or surgical environment could include oneor more of, “blood pressure,” “diabetes,” “ejection fraction,” and/orother statically significant parameters. In such an environment, one ormore of these names and/or one or more other, different names might besupplied in response to this request. Examples of parameter types thatmay be supplied in response to a request include (not necessarily incorrespondence to the example parameter names) “binary,” “enumerated,”“positive integer,” “decimal”, etc. An example of a possible limit is arange from 0 to 120, which might, for example, be a limiting rangeappropriate for an “Age” parameter name, and an example of a list ofpossible input values is “Male, Female” for a “Sex” parameter name. Theparameter types and lists of possible input values can be used to selectappropriate input formats when the model is used (e.g., a dropdown listfor a parameter name having an associated list). The limits can be usedto check the “sanity” of input values and to determine an appropriatecorrective action when an inappropriate value is entered for aparameter, e.g., the value “250” for “Age.”

In some configurations, the request for model parameters is sent via anXML Web Service for programmatic access. In configurations in which therequest is sent via an XML Web Service, the request is not necessarily“displayed” as such. More specifically, a Remote Modeling Service isprovided that uses a Web Method entitled “GetModelParameters” thattakes, as its argument, a unique model identifier for the model ofinterest. The GetModelParameters method returns a listing, which may bea complete listing, of parameters required by that model. In someconfigurations, a GetParametersForRequestedModels accepts a plurality ofunique model identifiers as its argument and returns a listing ofparameters, without redundancy, of parameters required by those models.Also, in some configurations, the listing is returned together with typeand/or validation information for some or all of the parameters requiredby the model. The entire request is communicated in some configurationsover SOAP. Data collected corresponding to the parameters requested andin response to the request is returned to the server, in someconfigurations, via the XML web service. The returned data is then usedfor execution in the requested models, and results of the execution arereturned to the requesting client over the XML web service.

In some configurations, the instructions for step 206 do instruct theoutcome type request and the request for one or more regression modelparameter names, etc., to be displayed. An example of an outcome typescreen used in some configurations is shown in FIG. 5 and an example ofa parameter name screen is shown in FIG. 6. At step 207, someconfigurations of the present invention also display a visual selectionof at least some of the parameter names along with a visual selection ofmathematical transforms for each displayed name. The user selects atransform, which can include, for example, logarithms, exponentials,squares, square roots, and other transformations, so that thesefunctions of the values later used for each corresponding variable nameare used in the regression equation instead of the variables themselves.In some configurations of the present invention, transformations arerendered on a display screen in a recognizable form, for example, astandard mathematical form. For example, in some configurations, if apower transformation is selected for a parameter, say, the square of“Age,” the display for this transformation is rendered as “Age²”wherever appropriate. In some configurations, scripting oftransformations is made available for custom, unique, or complextransformations that are not otherwise provided for selection.

At step 208, these names, parameter types, parameter limits, and/orparameter transforms are accepted by processor 110. At step 210 (whichmay be combined with step 208), values of the coefficients (and/or othertypes of values, if needed by the model) for the selected regressionmodel type and outcome are obtained. As used herein, the terms“coefficients” and “coefficient values,” unless otherwise explicitlyspecified, are intended to include within their scope not onlycoefficients, but also any constant or other terms that may be necessaryfor a model. Such terms may include, for example, an intercept term, amean square error term, and/or a number of degrees of freedom term.Thus, even though model forms are built from “standard components,” insome configurations they may be considered as fluid forms rather thanpredefined static structures into which coefficient values are simply“dropped.” For example, the number of terms used from a model form mayitself be defined by defining the number of degrees of freedom term. Inaddition, “coefficient” data, as used herein, also includes, unlessexplicitly stated, data computed “on-the-fly” from one or more parentparameters (e.g., the data is computed as a function of anotherparameter that is retrieved from a database or requested as input). Forexample, some configurations allow the specification of or provideautomatic recognition of at least some “parent-child” parameterrelationships. For example, body mass index (BMI) can be defined as aconstant times height divided by the square of the weight of a person.Thus, if one model uses BMI as a variable, and BMI is not available in adatabase but both height and weight are available, some configurationsof the present invention will automatically calculate BMI from heightand weight without issuing a separate request for BMI. If, in thisexample, a previous value of BMI exists for some reason and differs fromthe automatically calculated value, some configurations of the presentinvention display an alert icon next to the BMI parameter on the datainput form.

In some configurations, the coefficient values are obtained byrequesting that the values be input by a user at first client computer112. In other configurations, these values are obtained by instructionsto processor 110 to run a regression analysis on data obtained from adatabase 140, which may be a local database stored in computer 104 or adatabase accessible via a network such as network 113. A list comprisingthe outcome, associated coefficients and accepted names, types, and/orlimits for variables are stored in a memory (e.g., memory 106, asecondary storage unit, or even a register of the processor) of servercomputer 106 for later use at step 212. (The term “later use” isintended to be interpreted broadly and can include, for example, use aspart of the running of a stored model at a later date, use as part of aself-contained PDA version of the application, or use by anon-registered user who approached the application through the web to doa ‘one-off ’ run of a model.) In some configurations in which a visualselection of mathematical variable transforms has been selected, anindication of the selected mathematical variable transforms andcorresponding variable names is also stored in the memory. Someconfigurations also update β-weights and covariance matrices that arestored for the model.

In some configurations, derived parameters (for example, “Age Under 65,”which is derived from “Age,” or “BMI,” which is derived from “Height”and “Weight”) are stored, even though not explicitly collected from auser. The storing of “derived” regression model parameters and/or“derived” parameter names, unless otherwise explicitly stated, isconsidered consistent with and within the scope of “storing an acceptedlist of one or more regression model parameters and parameter names.”

In some configurations, the procedure represented by flow chart 200 (orby variations thereof such as those described herein) represents theinput of a regression model specification for (for example) a specifictreatment and a specific outcome. The procedure represented by flowchart 200 (or its variations) can be repeated to add further regressionmodel specifications representing different combinations of variablesfor different procedures, such as medical or surgical procedures. Someconfigurations further allow the editing and/or deletion of storedmodels. The parameters of regression model specifications are alsoreferred to herein as variables and/or functions of variables, such asin regard to steps in which values to the variables or functions ofvariables are assigned.

Referring next to flow chart 300 of FIG. 7, server module 150 alsocontains instructions configured to instruct processor 110 to allow auser (usually, but not necessarily different from the user at firstclient computer 112) at second client computer 126 to log into theserver module. The user at second client computer 126 is able to selectone or more of the stored regression model specifications, input datafor the stored models, run a regression analysis and be presented withresults of the regression analysis or analyses. More particularly, atstep 302, processor 110 is instructed to display a selection (forexample, a visual selection) of previously stored regression modelspecifications on second user display device 136 by communicating datavia network 113 with a program (e.g., a web browser) running on secondclient computer 126. An example of a display that provides a visualselection of previously stored regression model specifications is shownin FIG. 8. A user at second client computer 126 uses second user inputdevice 138 to indicate a selection of one or more of the displayedregression model specifications, and this selection is communicated toprocessor 110 via network 113, and processor 110 is instructed to acceptthis selection at step 304. At step 306, the server module determinescommon regression module variables (for example, those corresponding tothe same parameter name) between the one or more selected, storedregression models so that redundant requests for variables can beeliminated or reduced. The elimination of redundant requests forvariables (i.e., requesting the same variable more than once when aplurality of stored regression model specifications is selected by theuser) is also referred to herein as a “reduced-redundancy” request forvariables. In some configurations, derived values from parent variablesare also computed, so that data redundancy (not just model variableredundancy) is reduced or eliminated. For example, if one model requires“Age” as a continuous variable, and another model requires “Age lessthan 65” as a binary variable, some configurations of the presentinvention will not render the second parameter (i.e., “Age less than65”) for input, because it is redundant. Instead, these configurationsautomatically calculate the value of “Age less than 65” based upon thesupplied “Age” value. (In this example, “Age less than 65” is input as amathematical or other type of expression from which its dependence upon“Age” can be determined rather than as a separate, independent binaryvariable when the model is defined, so that its dependence upon thevariable “Age” can be recognized.)

Thus, at step 308, the server module displays a request or requests forvariables that correspond to the selected regression modelspecifications. An example of such a displayed request is shown in FIG.9. The server module accepts the collected variable data at 310 (whichmay also include an identification of a person or object to which thevariables apply) and runs the selected regression model specifications.At step 312, results of the selected regression model specifications aredisplayed. An example of such a display is shown in FIG. 10. Thedisplayed results can also include a representation of a statisticalrange, such as a visual representation in some configurations. Also insome configurations, at step 314, processor 110 is instructed to usecustomizable content (for example, letterhead data), previously storedin a memory or database accessible to processor 110 and optionallyincluding return addresses, logos, etc. to print the results or to causethe results to be printed. (Customizable content is not limited toletterhead, need not include the letterhead, and can include, in variousconfigurations, all or any portion or portions of the entire report. Byway of non-limiting example, the report can be customized in someconfigurations to be “informed consent” or similar type of document. An“informed consent” document can, in some configurations, includecustomization to provide applicable terms under which treatment isprovided, disclaimers, and/or signature lines for patients and/oranother person or persons authorized to provide consent for treatment.Other types of reports can include signature lines for a patient or hisrepresentative or guardian to sign confirming, for example, that therisks involved in undergoing or not undergoing a certain course oftreatment have been explained and understood.) The server module caninclude instructions to print different letterheads (or other types ofdocuments) depending upon credentials (e.g., login information) providedby the current user. Also, in some configurations, instructions toprocessor 110 at step 316 cause processor 110 to store variables applyin a memory or database (such as memory 104) accessible to processor110. Instructions in the server module are provided to instructprocessor 110 to provide default values for the stored variables whenoutcome results of the same or different type are requested for thatperson or object at a different time.

More particularly in some configurations, at runtime, collected modelparameters (i.e., user inputs) are serialized into a Predictor Vector:

$x^{\prime} = {\begin{pmatrix}x_{1} & x_{2} & \ldots & x_{r}\end{pmatrix} = \left\{ \begin{matrix}\begin{pmatrix}1 & z^{\prime}\end{pmatrix} & {Intercept} \\z^{\prime} & {{No}\mspace{14mu} {Intercept}}\end{matrix} \right.}$

where z is the transformed response vector, each of whose elementscorrespond to a single data item (e.g., systolic blood pressure).

The Linear Predictor is then calculated as

{circumflex over (η)}′=({circumflex over (72)}_(1 {circumflex over (η)})₂ . . . {circumflex over (η)}_(s))=(x′(e ₁ ·b)x′(e ₂ ·b) . . . x′(e _(s)·b))

in which each e_(i) is an extraction vector derived from the appropriateExtraction Matrix:

$\begin{matrix}{E = \begin{pmatrix}e_{1}^{\prime} \\e_{2}^{\prime} \\\vdots \\e_{s}^{\prime}\end{pmatrix}} & I\end{matrix}$

where each row e_(i)′ identifies the relevant sub-vector of b and thesub-matrix of V needed to predict the i^(th) independent parameter ofthe predictand. The extract form of E is dependent on the model type(see Table 1).

TABLE 1 Extraction Matrices Model Type Intercept No Intercept Linear1_(1x(r+1)) 1_(1xr) Generalized Linear 1_(1x(r+1)) 1_(1xr) CumulativeMultinomia 1 [I_(sxs) 1_(sxr)] [I_((s−1x(s−1)) 1_((s−1)xr) GeneralizedMultinomia 1$\left\lbrack \frac{I_{sxs}\mspace{14mu} \ldots \mspace{14mu} I_{sxs}}{r + {1\mspace{14mu} {times}}} \right\rbrack\quad$$\left\lbrack \frac{I_{sxs}\mspace{14mu} \ldots \mspace{14mu} I_{sxs}}{r\mspace{14mu} {times}} \right\rbrack\quad$Proportional Hazards N/A 1_(1xr)

In the table above, I denotes the identity matrix (i.e., a matrix withall diagonal entries=1 and all off-diagonal entries=0), and 1 denotes amatrix of all 1s.

The Prediction Vector containing the outcome point-estimate(s) is thengiven by

{circumflex over (μ)}=g ⁻¹ ^(({circumflex over (η)}))

where g⁻¹ is the appropriate inverse link function (see Table 2). TheLinear Predictor Variance Vector is given by

{circumflex over (σ)}′²=(x′(e ₂ ·V)x x′(e ₂ ·V)x . . . x′(e _(s)·V)_(x))

from which confidence intervals can be calculated as

({circumflex over (μ)}_(LO), {circumflex over (μ)}_(HI))=g⁻¹)({circumflex over (η)}±F _(v) ⁻¹(1−α/2)√{square root over({circumflex over (φ)}²+{circumflex over (ρ)}²)})

where F_(v) ⁻¹ is the inverse t-distribution function with v degrees offreedom at confidence level α and {circumflex over (φ)}² is the meansquare error ({circumflex over (φ)}² defined only for linear models).

TABLE 2 Inverse Link Functions Identity μ = η Log μ = e^(η) Logit μ =(1 + e^(−η))⁻¹ Probit μ = Φ(η) Power $\mu = \left\{ \begin{matrix}\eta^{1/\lambda} & {{{if}\mspace{14mu} \lambda} > 0} \\e^{\eta} & {{{if}\mspace{14mu} \lambda} = 0}\end{matrix} \right.$ Complementary log-log μ = 1 = e^(−e) ^(η)Cumulative logit^(τ) π_(i) = (1 + e^(η−i))⁻¹ i = 1, . . . , s Cumulativeprobit^(τ) π_(i) = Φ(η_(i))* i = 1, . . . , s Cumulative complementaryπ_(i) = 1 − e^(−e) ^(ηi) i = 1, . . . , s log-log^(τ) Generalized Logit$\mu_{i} = \left\{ \begin{matrix}{e^{\eta \; i}\left( {1 + {\sum_{j = 1}^{s}e^{n_{j}}}} \right)}^{- 1} & {{i = 1},\ldots \mspace{14mu},s} \\\left. {1 + {\sum_{j = 1}^{s}e^{\eta_{j}}}} \right)^{- 1} & {i = {s + 1}}\end{matrix} \right.$ Proportional Hazards μ = S_(j) ^(e) ^(η) ^(τ)Formulti-state models, $\mu_{i} = \left\{ \begin{matrix}\pi_{i} & {i = 1} \\{\pi_{i} - {\sum\limits_{j = 1}^{i - 1}\pi_{j}}} & {{i = 2},\ldots \mspace{14mu},s} \\{1 = {\sum\limits_{j = 1}^{i - 1}\pi_{j}}} & {i = {s + 1}}\end{matrix} \right.$ *Φ is the standard normal distribution function

In various configurations of the present invention, model building isgreatly facilitated by a visual editing environment that providesdynamic on-screen instructions and rendering of prediction formulae aswell as robust validation services. Parameters are identified that areused in a prediction formula. New parameters can be created de novo orselected from an existing parameter library (thereby allowing forstandardization of parameter definitions across all models).

Once parameters have been defined, one or more data transformations canbe assigned to each. A broad array of built-in transformation types areavailable to the user, and custom transformation types can be readilydefined via scripting, allowing configurations of the present inventionto handle a wide range of complex formulae.

Main effects and interaction terms derived from the input parameters andtheir transformations can be derived in some configurations of thepresent invention, and regression coefficients for calculating pointestimates for outcome of interest and optional covariance estimates canbe provided for computing confidence intervals.

Once regression model specifications have been built and deployed,health care providers (or, in other environments, other individuals) canreadily access them through an integrated and customizable portalinterface using a variety of web-enabled devices. Dynamically generateddata entry screens are provided based on the variables required by theselected model(s). For example, in medical environments, data forpatients can either be entered de novo or retrieved from a patientinformation system, which can be readily integrated with existingclinical information systems via XML web services support.

Some configurations of the present invention render model outputs in avariety of graphical and non-graphical formats, including solid barplots, gradient bar plots, whisker line plots, pie charts, and/ordigital LED-style displays, which can be user-selectable. Output frommultiple models can be grouped onto a single plot to facilitateinter-model comparisons (for example, stroke risk with angioplastyversus bypass surgery). In addition, some configurations allow a user tocustomize the output plot style, the selection of models to include in afinal output and the display of confidence intervals (when modelcovariance data has been provided). In various configurations, users canprint outcome plots using customizable report templates in order togenerate documents such as patient educational materials and informedconsent sheets. Also in some configurations, outcomes researchers cancustomize report and page content using a built-in Microsoft Word®-likeinterface or by editing HTML code. A feature rich set of portal contentmodules, including workgroup directories, discussion threads, anddocument repositories can be provided in server module configurations ofthe present invention to allow outcomes research groups to easilycreate, manage, and build their own collaborative web sites.

It will thus be appreciated that configurations of the present inventioncan be used to handle various aspects of data collection, validation,storage/retrieval, and processing, thereby freeing outcomes researchersfrom intricacies of programming and networking.

While the invention has been described in terms of various specificembodiments, those skilled in the art will recognize that the inventioncan be practiced with modification within the spirit and scope of theclaims.

1. A method for generating interfaces configured to receive patientvariable data to be processed by mathematical health outcome predictionmodels, the method comprising: receiving, by a computer system, anidentification of a first plurality of mathematical health outcomeprediction models; accessing the first plurality of mathematical healthoutcome prediction models; dynamically generating, by the computersystem, an interface based at least in part on respective definitions ofthe first plurality of mathematical health outcome prediction models,wherein the dynamically generated interface is configured to requestpatient information corresponding to patient variables of the firstplurality of mathematical health outcome prediction models, wherein thedynamically generated interface is configured by the computer system toreduce redundancy in the requested information at least in part byidentifying one or more variables that may be shared among the firstplurality of mathematical health outcome prediction models based atleast in part upon the respective definitions of the first plurality ofmathematical health outcome prediction models.
 2. The method as definedin claim 1, wherein the interface retrieves patient information from apatient information system.
 3. The method as defined in claim 1, whereinthe interface obtains patient information from a patient informationsystem using an XML web service without displaying the request.
 4. Themethod as defined in claim 1, wherein the interface is provided fordisplay to a user, and patient information entered by the user isreceived at the computer system.
 5. The method as defined in claim 1,wherein the interface is configured to obtain patient informationrelated to a patient's gender, age, and history of diabetes.
 6. Themethod as defined in claim 1, the method further comprising providing auser interface for inputting a plurality of regression modelspecifications related to medical treatment and outcome.
 7. The methodas defined in claim 1, the method further comprising: (a) inputting aregression model specification related to providing health relatedpredictions; and (b) repeating (a) a plurality of times to obtain andstore a plurality of the regression model specifications.
 8. The methodas defined in claim 1, the method further comprising: (a) inputting aregression model specification related to providing health relatedpredictions; (b) repeating (a) a plurality of times to obtain and storea plurality of the regression model specifications; and using results ofthe plurality of regression model specifications to create an informedconsent document and/or patient educational material including adiscussion of treatment risks and benefits.
 9. The method as defined inclaim 1, the method further comprising: (a) inputting a regression modelspecification related to providing health related predictions; (b)repeating (a) a plurality of times to obtain and store a plurality ofthe regression model specifications; and using results of the pluralityof regression model specifications to create a clinical decision aid.10. The method as defined in claim 1, the method further comprising: (a)inputting a health outcome prediction model specification related toproviding health related predictions; (b) repeating (a) a plurality oftimes to obtain and store a plurality of health outcome prediction modelspecifications related to providing health related predictions;executing a plurality of health outcome prediction model specificationscorresponding to the first plurality of mathematical health outcomeprediction models; and providing for display results of execution of thehealth outcome prediction model specifications.
 11. The method asdefined in claim 1, the method further comprising: (a) obtaining ahealth outcome prediction model specification related to providinghealth related predictions; (b) repeating (a) a plurality of times toobtain and store a plurality of health outcome prediction modelspecifications related to providing health related predictions;executing a plurality of health outcome prediction model specificationscorresponding to the first plurality of mathematical health outcomeprediction models; and providing for display results of execution of thehealth outcome prediction model specifications.
 12. A computer systemconfigured to facilitate the use of health outcome prediction models,the computer system comprising: at least one computing device; memorycoupled to the at least one computing device, the memory havinginstructions stored therein that when executed by the at least onecomputing device are configured to cause the computer system to performoperations comprising: receiving an identification of a first pluralityof mathematical health outcome prediction models; accessing the firstplurality of mathematical health outcome prediction models; dynamicallygenerating an interface based at least in part on respective definitionsof the first plurality of mathematical health outcome prediction models,wherein the dynamically generated interface is configured to requestpatient information corresponding to patient variables of the firstplurality of mathematical health outcome prediction models, wherein thedynamically generated interface is configured to reduce redundancy inthe requested information at least in part by identifying one or morevariables that may be shared among the first plurality of mathematicalhealth outcome prediction models based at least in part upon therespective definitions of the first plurality of mathematical healthoutcome prediction models.
 13. The computer system as defined in claim12, wherein the interface is configured to analyze patient informationfrom a patient information system.
 14. The computer system as defined inclaim 12, wherein the interface is configured to obtain patientinformation from a patient information system using an XML web servicewithout displaying the request.
 15. The computer system as defined inclaim 12, the operations further comprising providing the interface fordisplay to a user, and receiving patient information entered by the uservia the interface.
 16. The computer system as defined in claim 12,wherein the interface is configured to obtain patient informationrelated to a patient's gender, age, and history of diabetes.
 17. Thecomputer system as defined in claim 12, the operations furthercomprising providing a user interface for inputting a plurality ofregression model specifications related to medical treatment andoutcome.
 18. The computer system as defined in claim 12, the operationsfurther comprising: (a) inputting a regression model specificationrelated to providing health related predictions; and (b) repeating (a) aplurality of times to obtain and store a plurality of the regressionmodel specifications.
 19. The computer system as defined in claim 12,the operations further comprising: (a) inputting a regression modelspecification related to providing health related predictions; (b)repeating (a) a plurality of times to obtain and store a plurality ofthe regression model specifications; and using results of the pluralityof regression model specifications to create an informed consentdocument and/or patient educational material including a discussion oftreatment risks and benefits.
 20. The computer system as defined inclaim 12, the operations further comprising: (a) inputting a healthoutcome prediction model specification related to providing healthrelated predictions; (b) repeating (a) a plurality of times to obtainand store a plurality of health outcome prediction model specificationsrelated to providing health related predictions; executing a pluralityof health outcome prediction model specifications corresponding to thefirst plurality of mathematical health outcome prediction models; andproviding for display results of execution of the health outcomeprediction model specifications.
 21. The computer system as defined inclaim 12, the operations further comprising: (a) obtaining a healthoutcome prediction model specification related to providing healthrelated predictions; (b) repeating (a) a plurality of times to obtainand store a plurality of health outcome prediction model specificationsrelated to providing health related predictions; executing a pluralityof health outcome prediction model specifications corresponding to thefirst plurality of mathematical health outcome prediction models; andproviding for display results of execution of the health outcomeprediction model specifications.
 22. The computer system as defined inclaim 12, further comprising a server including a network interfaceconfigured to receive at least a portion of the patient variables fromone or more client systems.
 23. A non-transitory computer readablestorage medium having instructions stored thereon that, if executed by acomputing system, cause the computing system to perform operationscomprising: receiving an identification of a first plurality ofmathematical health outcome prediction models; accessing the firstplurality of mathematical health outcome prediction models; dynamicallygenerating an interface based at least in part on respective definitionsof the first plurality of mathematical health outcome prediction models,wherein the dynamically generated interface is configured to requestpatient information corresponding to patient variables of the firstplurality of mathematical health outcome prediction models, wherein thedynamically generated interface is configured to reduce redundancy inthe requested information at least in part by identifying one or morevariables that may be shared among the first plurality of mathematicalhealth outcome prediction models based at least in part upon therespective definitions of the first plurality of mathematical healthoutcome prediction models.
 24. The non-transitory computer readablestorage medium as defined in claim 23, wherein the interface isconfigured to retrieve patient information from a patient informationsystem.
 25. The non-transitory computer readable storage medium asdefined in claim 23, wherein the interface is configured to obtainpatient information from a patient information system using an XML webservice without displaying the request.
 26. The non-transitory computerreadable storage medium as defined in claim 23, operations furthercomprising providing the interface for display to a user, and receivingpatient information entered by the user via the interface.
 27. Thenon-transitory computer readable storage medium as defined in claim 23,wherein the interface is configured to obtain patient informationrelated to a patient's gender, age, and history of diabetes.
 28. Thenon-transitory computer readable storage medium as defined in claim 23,the operations further comprising providing a user interface forinputting a plurality of regression model specifications related tomedical treatment or condition and outcome.
 29. The non-transitorycomputer readable storage medium as defined in claim 23, the operationsfurther comprising: (a) inputting a regression model specificationrelated to providing health related predictions; and (b) repeating (a) aplurality of times to obtain and store a plurality of the regressionmodel specifications.
 30. The non-transitory computer readable storagemedium as defined in claim 23, the operations further comprising: (a)inputting a regression model specification related to providing healthrelated predictions; (b) repeating (a) a plurality of times to obtainand store a plurality of the regression model specifications; and usingresults of the plurality of regression model specifications to create aninformed consent document and/or patient educational material includinga discussion of treatment risks and benefits.
 31. The non-transitorycomputer readable storage medium as defined in claim 23, the operationsfurther comprising: (a) inputting a health outcome prediction modelspecification related to providing health related predictions; (b)repeating (a) a plurality of times to obtain and store a plurality ofhealth outcome prediction model specifications related to providinghealth related predictions; executing a plurality of health outcomeprediction model specifications corresponding to the first plurality ofmathematical health outcome prediction models; and providing for displayresults of execution of the health outcome prediction modelspecifications.
 32. The non-transitory computer readable storage mediumas defined in claim 23, the operations further comprising: (a) obtaininga health outcome prediction model specification related to providinghealth related predictions; (b) repeating (a) a plurality of times toobtain and store a plurality of health outcome prediction modelspecifications related to providing health related predictions;executing a plurality of health outcome prediction model specificationscorresponding to the first plurality of mathematical health outcomeprediction models; and providing for display results of execution of thehealth outcome prediction model specifications.